I am very happy to have this opportunity to communicate with you in DEF CON.
I want to talk to you about the area of AI security that I have been studying.
Those of you who are going to the bathroom, don't worry, take your time.
This PPT will take a long time.
Next, I will briefly introduce our team.
We are the security lab under Baidu Security Department.
My team mainly studies the areas related to AI security, including AI model defense and AI model protection.
Baidu Security Department has opened a lot of open source software.
Here are a few of the most important ones.
One is the ADVBOX introduced this time.
It is actually a tool box for AI model security.
What Mr. Ma just introduced is the T-shirt that can be hidden.
Including the face changer that we talked about later,
it is all based on this tool box.
If you are interested, you can download this tool box and play with it.
The mainstream deep learning models should all support it.
It is also the only AI model attack box that supports PIDO.
The rest are MesaLog and MesaLink.
The other one is OpenRASP, which we have used a lot.
It is also developed by our team.
It is a next-generation WAF engine.
Next, I will briefly introduce myself.
My ID is Douge, and my real name is Liu Yan.
When I first came up with this name,
I had a simple understanding.
All the girls in the university are good at getting married.
All the boys in the university are not bad at getting married.
So I named myself Douge.
I am the person in charge of the AI model team.
I used to study a lot about the security of the outside world.
In recent years, I have been studying some areas related to AI security.
I have written some books for myself.
I am an experienced author.
I have read some books.
If you are interested, you can have a look.
Of course, our team is still recruiting.
If you are interested in joining our team to study the security of AI,
you can scan my QR code.
Today, we are going to talk about how to attack the cloud with a black box.
How do we understand this problem?
We know that in the early days,
our AI models were all local.
For example, some AI models on our mobile phones
can detect human faces and voices.
But training a model and operating a model
requires a lot of time and energy.
Therefore, the most popular way
is to put my AI model directly on the cloud.
This way, I can use a lot of mobile devices.
Many developers can also use our AI capabilities directly.
They do not need to deploy a set of AI framework locally.
Here are some simple examples.
For example, your audio box.
The content of what you say to the audio box
or other applications you have developed
can be analyzed using the cloud API.
You can analyze what content is in the picture
and what image is in the image.
This is a cloud application.
This is a cloud AI API.
Today, we are going to talk about
how to attack the cloud AI API
with a black box.
The general idea is that
for example, I want to throw a picture
into the cloud to predict.
I add a certain disturbance on the picture
to see if I can deceive the cloud API.
For example,
if I normally throw a picture of a cat,
it returns me a cat probability,
for example, 0.99.
If I carefully add some disturbance
on this picture,
it may turn into something else.
For example, a baked bread machine.
This is an approximate structure.
Next, we will briefly introduce
different attack modes.
In the attack AI model,
the simplest mode is the white box attack.
The so-called white box attack
is what I know about the entire AI model.
Whether it is its network structure
or even the parameters of each layer,
I know them all.
And I can visit its input without any restrictions.
This is a very important point.
Therefore, the white box attack
is relatively less difficult.
A little bit more difficult
is the so-called black box attack.
First of all, I don't know
what the model of my attack looks like.
I don't know its structure
or parameters.
The only thing I can do
is to visit its input without any restrictions.
Of course, this is a relatively broad condition.
What we can introduce later
is that when attacking the cloud,
it is actually a special black box attack,
which is even more difficult.
In addition to that,
I don't know the structure of the model
or its parameters.
I even have to go through
an unknown pre-processing process.
For example, a picture that I recognize
may be erased, stretched,
or a picture that has been taken out of the middle.
This will all happen.
There is also a very important point.
When I visit the cloud,
I visit it through the network.
Therefore, my visits to it
will also be limited.
Therefore, it is very difficult
to attack the cloud.
What will this cause?
It will give us a feeling of error.
I thought that if I put the model
on the cloud,
it would be much safer than putting it in the local area.
It seems to be logical.
But we will prove later
that this wrong sense of security exists.
By the way,
the picture just now
seems to be the picture
that Mr. Duan introduced
at the conference.
The meaning of this picture
is that as a intellectual,
he has a duty to expose the dark side of society.
Of course, this is a big deal.
For me,
as a security worker,
we actually have a duty
to expose the existence of loopholes.
Because you only accept the existence of loopholes
to be able to really push loopholes to repair.
Next, we will introduce
the form of black-and-white attack.
Because it is black-and-white,
we don't know the structure of the model,
we don't know the parameters.
Therefore, the most natural idea
is that we can only attack it
by constantly trying.
Therefore, the simplest and most common
way of black-and-white attack
is to make an appointment.
Our understanding of the structure of the model
comes from my continuous attacks
or my continuous visits.
During many visits,
I constantly understand the structure of the model
and even its parameters.
However, when I attack it,
I have to go through many visits.
Therefore, based on the attack of the visit,
I need a large number of visits
to master the structure of the model.
A simple example is that
if I want to generate a low resolution attack image,
I may need thousands of visits.
If I want to generate a high resolution image,
I may need
tens of thousands of visits
or even millions of visits.
Here is an example.
In a paper,
the success rate of the attack
is 95%.
However, every visit
requires 100,000 visits.
This leads to two problems.
One is that
the number of visits is so large
that the cloud API
limits the number of visits.
The other reason is that
it is very expensive.
Why?
Because every visit is charged.
Usually, a visit costs
tens of thousands of visits.
If a visit costs
tens of thousands of visits,
a visit costs
tens of thousands of visits.
Therefore,
such visits are slow
and cost-effective.
For example,
if you want to attack
10,000 or 10,000 photos,
you can imagine
how much it costs.
Therefore,
we think that
a visit costs
tens of thousands of visits.
However,
if you want to attack
10,000 or 10,000 photos,
you can imagine
how much it costs.
Therefore,
if you want to attack
10,000 or 10,000 photos,
you can imagine
how much it costs.
Therefore,
if you want to attack
10,000 or 10,000
It's a little bit smaller, but you can take a look at it.
This row of trees is the model A that I mentioned before,
which is the open source model that we already have in our hands.
We will generate our counter samples for this row of trees.
The horizontal row is the model that we are going to attack.
In fact, you can see that there are five models that we have in our hands,
and there are five models that we are going to attack.
We can see this row of shoes.
This is the probability that all of our values can be successfully identified.
The higher the probability, the more our attack will fail.
The lower the probability, the more our attack will succeed.
Because the purpose of my attack is to make your model identify errors.
You can see that the horizontal row, for example, the first row,
the source of the attack is resnet 152, and the target is also 152.
My attack success rate is close to 100%, which means that my recognition rate is 0.
I will repeat this point again.
The lower the value, the higher the success rate of your attack.
You can see that if it is my conscious model,
it is exactly the same as my attacked model.
Basically, it can be said that 100% of the attack can be broken.
But if we look at it diagonally,
we can see that it is the same as using Resonant 152 to attack other models.
Its recognition rate is also very low.
This actually reveals one point.
When we are learning to move or attacking,
the closer the structure of the model is,
the higher my attack success rate will be.
But what kind of problem does moving learning have?
As you can see, these open source models that I have in my hand,
all of them are doing image classification,
and they are all doing 1,000 imaginate classifications.
In fact, if I am also attacking this kind of image classification,
and I am also attacking this kind of imaginate,
in the same way, my attack success rate is very high.
But in real life,
for example, if I want to attack a picture review,
or a car review service,
I basically can't get so many open source models to attack.
So this is a must-have for learning.
I must be able to master enough open source models in my hand.
So this is a must-have for learning.
What is the other point?
Although you can see that the attack success rate of this table is relatively high,
but in the real process,
you can actually find that the efficiency of other models' attacks is not that high.
This effect is a more classic experimental effect.
In other words, moving learning has a great effect,
but in the actual work,
it does have some influence in it.
Next, I will talk about our attack method.
What is our ideal situation?
What is our ideal situation?
First of all, I can attack the cloud API.
On the other hand,
I hope that my attack task can be customized at will.
For example, I can attack cats and dogs,
or I can attack you to identify cattle and pigs,
or I can identify BMW and Mercedes-Benz.
I hope that my attack task can be changed.
Of course,
careful students can pay attention to
the topic we talked about before,
which is to attack another type of image recognition.
But in my national environment,
we have to talk about something that can be broadcast on TV.
We can't talk about something that can't be broadcast on TV.
So, in this example,
we are talking about how to attack
cats and dogs in image recognition.
In fact, in the same way,
we can also attack car recognition
and other image recognition.
The next point is to introduce our attack.
We have proposed a new attack method.
This method is based on a kind of replacement learning.
On the other hand,
we have an improvement point.
For replacement model learning,
we are based on a kind of R2DL technology.
How to understand this problem?
I have introduced it before.
Whether it is based on searching
or moving,
there are all kinds of
For example,
how to reduce the number of attacks.
If I have a model similar to the cloud,
I can't attack it directly.
My attack speed is very low.
So, there will be an idea of
this kind of replacement attack.
The first to propose a replacement attack
should be the 2018 World Final.
It will be more famous.
We also borrowed this idea
to use a replacement attack.
But our improvement point is
the training process of our replacement model.
In their paper,
it requires a lot of training data.
But in our case,
after our experiment,
we actually only get,
for example,
100 or even only 50 pictures.
We can restore a function
that is basically the same model as the cloud.
This benefits our R2DL technology.
Compared to the black box search,
it requires thousands of times,
hundreds of times,
or even millions of times of search.
Our search actually only happens twice.
One time is the process of marking.
For example,
after I have collected enough pictures in my hand,
I will first go to the cloud to search.
It will tell me
if this is a cat or a dog.
This is the first time.
This is the process of marking.
The second time is the process of my attack.
I attack the cloud directly
with the sample I generated.
So, on average,
we only search twice
for each picture.
So,
the other improvement is that
our attack is more universal.
It is not dependent on
having the same open source model
in your hand.
Of course,
we can see that
for example,
we have to complete
a thousand different image models
for Imaginate.
But for example,
we will introduce later
such as attacking a cat or a dog,
attacking a BMW,
and so on.
At present,
we do not have this open source model in our hands.
But based on our technology,
we can also attack.
This is our
relatively general
learning advantage.
OK.
Next,
I will introduce
our entire attack process.
The attack process is very simple.
There are two steps.
The first step is
the training of the replacement model.
The second step is
to generate the sample.
Next,
I will introduce
how we choose
a better replacement model.
There are thousands of high-rise buildings
together.
But
we still have to stand
on the shoulders of our predecessors.
So,
what we use is
the model of Imaginate
that has been pre-trained.
What is our general understanding?
It is that
the model of Imaginate
is relatively good
in terms of image separation.
It is also
feature selection.
So,
the four models
on the right
can be selected.
For example,
from
ResNet152
or
the following
are all fine.
These four
functions are similar.
You can choose
whatever you want.
We choose
the one in the middle,
which is ResNet152.
Then,
the whole process
of learning
is relatively simple.
We will simplify
our attack process
into two categories of problems.
For example,
a cat
or not a cat.
Similarly,
you can also use
for example,
you want to attack
an image
category
of car identification,
such as
BMW and Benz.
You can simplify it
into
BMW or not BMW.
Simplify its problem
is enough.
In addition,
because we are
a category 2 problem,
we can change
the output of our model
to 2.
We need to
improve the model
based on
ResNet152
which is
So,
we just need to
tweak the model.
However,
what we need to emphasize
is that
we need to tweak
every layer
of the model.
In fact,
it is not convenient
the characteristics
of the common model.
It only cares about
the last pre-connected layer.
This is a
normal
model prediction
process.
For example,
if I throw
a cat's picture
into the neural network,
it will output
two values.
One is the probability
that you are a cat,
and the other is
the probability of other teams.
Normally,
we can know that
it should be a cat
and the probability is 0.99.
In our model training process,
another point is
very different
from the general
physical learning,
is that
we do not take
a large number of pictures
to the cloud
to check the shadow,
so that we can train
a model locally.
In fact,
we just need to take
our attack sample.
For example,
I want to take
my 50 pictures
of cats
to generate
an attack sample.
Then,
I just need to take
these 50 cats
to train.
Instead of taking
1,000 cats
and 1,000 dogs
in other training methods,
we can restore
a complete model locally.
This is a different
place for us.
OK.
The next point
we will introduce
is the whole
training process.
Here,
we introduce
the bidding process.
First,
we initialize
our database.
We solve
the problem of a cat
with a certain number
of pictures of cats.
The other part
is the picture
that is not a cat.
It can be an elephant,
a pig,
a dog,
or even a car.
It doesn't matter.
This way,
we get
an initialized database.
Then,
we take
this initialized database
to the cloud
to search.
The cloud
will return
a result to me.
It tells me
that from the perspective
of the cloud,
this is a cat
and not a cat.
So naturally,
we use this result
to mark
our image.
The so-called mark
is to connect
our image
with an object.
In this way,
we complete
the bidding process.
After the bidding process,
we get
an initialized database
called
SEP.
We use
this initialized database
to restore
a model
that is close
to the cloud
in our local area.
This is the process
of completing
a local training.
This process is quite annoying.
You don't have to worry about it.
Let me introduce
what it means.
In a neural network,
there are usually
a lot of parameters.
There may be
tens of thousands,
millions,
or even hundreds of millions
of parameters.
One parameter
that is considered
by the public
is that
if the number
of parameters
in my model
is smaller
and the size
of the model
is smaller,
then my roughness
will be better.
Why?
For example,
the so-called roughness
means that
if my error
changes slightly,
it won't affect
my prediction
too much.
In the same way,
if the parameters
in my model
are relatively
small
and relatively
small,
then the interference
in my model
will also be
relatively small.
So,
my model
will also be
more rough.
This is what
everyone thinks.
So,
the meaning
of our formula
is that
in addition to
training it,
I will also
approximate it
to make its parameters
as small as possible.
The last part
of this formula
actually shows that
I will approximate
the size
of the parameters
in my model.
The way to do this
is usually called
the L2 punishment
or the L1 punishment.
It doesn't matter.
The so-called L1 punishment
actually makes
the size
of each parameter
in our model
smaller.
I hope that
the number
of models
in our model
is as small as possible.
The L2 punishment
is actually
to approximate
from an OG leader.
You don't have to
think too much about this.
In general,
the principle is that
we add an approximate
number
in the process
After adding an approximate number,
we can make
the parameters
of our model
as small as possible.
Another point
is that
our R2DL
has played a role
In fact,
what we used
is a function
of automatic
It is not the last day
of the project
that tells you
that the project
is not good
and what you need
to do to improve it.
What would a good boss
do?
At a critical moment
in a project,
for example,
in demand analysis,
development,
and testing,
it will show you
the key times
and see
if you can run
the course.
This is a simple
understanding.
When we train
neural networks,
there will also be
this problem.
How to understand it?
This green dot
is not very obvious.
For example,
in a normal
training model process,
I only care about
the output of your model
and the deviation
of my expected prediction value.
For example,
if a cat is thrown in,
you may predict that it is a dog.
I will tell you
that your boss
predicted it wrong.
You give me
to change your parameters.
This is the most common process.
And what we do
is that
in some important stages
of model training,
I will correct
the error of your model.
When it is realized,
it is actually
the most important
output points
of my neural network.
I will check
the deviation
of your model
and the value
of your model.
By doing this,
I can train
my model better.
The intuitive feeling
is that
others may need
1000 cats
to train
a model
like a cloud
locally.
But we may only need
50 cats
to train.
Because our boss
is good enough.
Just like Mr. Ma.
Just now,
I introduced
how to train
a better model
locally.
Of course,
after we finish this step,
we can think that
we have already trained
a model
that is almost
completely similar
to the cloud function
What is the second step?
We transform
the process of attacking
a black box
into a white box.
We attack this model
locally
to generate
a counter sample.
Next,
we will introduce
some improvements
in the white box
of our algorithm.
We know
that when we generate
some interference
on the picture,
or when we add
some disturbance,
its classification
will be wrong.
At the same time,
if we change
this picture
to a pixel point,
it will not affect
the difference
between people.
For example,
if it is still a cat,
if we change
a few pixel points,
our model
will be identified
as a dog.
The important point
is that
first,
if we change
some pixel points,
the machine
will be wrong.
The second point is that
people
will not be affected
by this.
It is still a cat.
Now,
the main point is
how to make
our machine
identify the error.
As you can see,
in our definition
of this loss function,
what we dislike
is that there are
more formulas.
I will try to use
the language
that everyone can understand
to make it more transparent.
As you can see,
the first one,
classLoss,
is actually
our definition
of this loss function.
We hope that
after the modification
of the picture,
we can try
to cheat the model
of this loss function.
It is a classification error.
The second point
is that
we have an improvement
here.
At the same time,
we also hope that
in the output of
our neural network
collection,
its deviation
will be as large as possible.
Later,
we will introduce
the significance
of this larger deviation.
Here,
we introduce
the loss function.
In fact,
its main purpose
is to make
our classification
to be wrong.
And the logic
of our futureMap
actually hopes that
our attack sample
will have better
resolution
resolution.
How to understand
this?
The second point
is that
FusionLoss hopes that
we can cheat
model A
and at the same time
cheat model B
at the same time
OK,
you can see
the point
behind us.
This is a
common
example
of a neural network
collection.
We can see
a neural network
and divide it
into several layers.
The first layer
we can define
is a low-level
collection layer.
We can call it
low-level
feature.
It collects
low-level
features.
You can see
some edges
or some lines.
This is what
was done
in the first layer
It should be said
that it was done
in the first few layers.
What is the second layer
doing?
It collects
a slightly
higher layer
of features.
For example,
some deep layers
or even
some eyes
or some edges.
It is based on
this kind of
rough features
and collects
some higher features.
At the third layer
you can see
that even
a complete
outline of a cat
can be seen.
So,
at the third layer
it collects features
that can be considered
high-level
features.
So,
when we attack
the neural network
we hope
to rely on
the output
which is
than the original
difference.
The higher the level
the higher the probability
between the cat
and me.
So,
the higher the probability
of attacking other models
the higher the probability
So,
in this
we also
introduced
a little bit
about the
of different layers
and the
value of different layers.
In other words,
the further we go
the more we pay
attention to it.
The further we go
the lower
the level of attention
will be.
If the model
relies on the back
the difference
will be greater
and its
potential
will be stronger.
We will introduce
this later
through numbers.
This is
a specific definition
of the
of the
of the
of the
of the
of the
of the
of the
of the
of the
of the
the
of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of the of
There are two advantages.
First of all, Imagine.net is an open source database.
Everyone can get it.
This database is open source and fair.
I didn't just find a hundred pictures.
What are the other advantages?
It comes from the Imagine.net test database.
It also says that this data itself is not used to train open source models.
So this data is both fair and will not affect my model itself.
Another point is that in order to adapt to different models,
I resized all the pictures into 224x224x13,
and read them in the form of RGB.
This is a more common practice.
Then we measure the effect.
What is it used for?
It's for the escape rate.
The so-called escape rate is, for example,
normally, I throw a picture into the machine learning model.
It might give me a bunch of lists.
For example, it's 10 categories or 100 categories.
Each category has a different probability.
Usually, for example,
the previous model,
the probability ranges from large to small.
Top 1 is the correct answer if the first one is correct.
It's called the accuracy of top 1.
If there is only one correct answer from the first to the third place,
we think this is the accuracy of top 3.
For example, I still throw in a picture of a cat.
If it returns the result as a cat,
the probability is 0.99, and it is the highest.
This is the correct answer for top 1.
If I throw in a cat,
it will give me the first place is a pig,
the second place is a cat,
and the third place is a dog.
We think the probability of top 3 is still correct.
Here we use the most rigorous method.
We only care about the first result it returns.
If the first result is not a cat,
we think our attack is successful.
So we will use the error recognition rate of top 1
as our measurement index.
We define it as the escape rate.
Then we use the REST 152 model itself for training.
In fact, we introduced REST 152,
REST 101, and the four models in the back.
As long as the accuracy is higher, it can be used.
Then based on the T-shaped model that we have trained now,
we have launched a parallel simulation model.
We have introduced the PCD model and the FII-L model.
We usually think that the PCD model is the most powerful attack model
in the current white-collar attack algorithm.
So we use it as a basis.
On the other hand,
there is another one that we think is more powerful in attack,
which is the base attack.
We will see the result later.
Another indicator is that it is used more in the field of image recognition.
For example,
we mentioned that the contrast sample has a very important point.
The difference between the images I have created and the original images is not big.
The human eye can receive it.
This is a very sensitive indicator.
How do we lighten this indicator?
The most common way is to use the heart rate meter.
By using the heart rate meter,
we can compare the difference between the original picture and the sample.
The higher the heart rate meter,
the lower the image quality and the image quality.
The general understanding is that
if the heart rate meter is 20 times higher,
the human eye can accept this kind of change.
If it is lower than 20 times,
the human eye can already notice the obvious loss of focus.
However,
in our practice,
we think that 20, 19 or more can be accepted.
It is not too strict.
The ideal situation is that it is more than 20.
If it is more than 20,
its focus will be relatively small.
Another indicator is to measure the image.
This is SSIM.
This can be obtained through the normal library.
It is used to measure the similarity between two pictures.
Usually,
these two indicators will be used simultaneously.
The general SSIM is based on 0 to 1.
The larger the value,
the better the similarity.
Generally, it is more than 0.5. It can be accepted.
This is the result of the test
of four non-Chinese cloud platforms
that are mainstream in the international market.
Because in China,
some foreign platforms are more reliable.
Otherwise,
I will be a pioneer.
As you can see,
what is realized
is the attack algorithm
that we proposed.
Then,
what corresponds to the virtual line
is the result
of our attack PDE algorithm.
This is a standard algorithm.
As you can see,
the following 1, 2, 3, 4, and 5
are my attacks.
As my attacks increase,
the success rate of our attacks
is getting higher and higher.
When the attack is between 7 and 8,
mainstream platforms can be
more than 90%.
This is Taoyue.
This is the result of our attack.
At the same time,
we also need to pay attention
to the quality of our images.
As you can see,
the quality of our images
is higher than that of PGD.
However,
we are more than 20%.
Therefore,
both of us are satisfied
with the quality of our images.
The blue one
is our attack sample.
The green one is PGD.
The similarity of our images
is significantly higher
than the traditional attack method.
In other words,
when the quality of our images
is acceptable,
the similarity of our images
will be higher.
This is the result
of our comparison with PGD.
Another way to recognize
the better results is
when we use PGD.
What is PGD?
For example,
if I have 5 or even 10 models
in my hand,
I will use them one by one
to create a sample.
We understand that
this sample is more efficient
because I have 10 different models
that can attack successfully.
We think that the success rate
of attacking other models
will be higher.
In the case of a pure black box,
that is,
when there are no images
in the preview,
the attack effect is very good.
Therefore,
we also use it as a basic model.
However,
the actual effect
is very unexpected.
As you can see,
the attack success rate
is very low.
Basically,
the attack success rate
is less than 50%.
No matter how I increase
the number of attacks,
the number of attacks
is not too long.
Because this kind of
attack is usually done
in an optimized way.
Only one of the models
can attack successfully.
What is the difference
between the two models?
Compared to the pure black box,
there is a very important process
in which the attack effect
is set randomly
to a certain length
and size,
and then it will be removed.
Even if it is randomly drawn,
in the case of optimized
attack method,
some of the disturbances
generated are easily
destroyed by this kind of
image art.
This also explains
why attacking the cloud
will be more complicated
than the attack effect.
In the case of other models,
such as Google,
Safari,
and Amazon,
the attack effect is relatively poor.
We think it is because
the image art processing
is relatively strong
in these three cloud processing
models.
You can see that
in our attack chart,
the attack effect
is relatively good.
For example,
in the case of 7 and 8,
the attack effect
is close to 90%.
This shows that
when we use this kind of
long-range attack
to deal with image art,
our Lu Bang is strong enough.
After talking so much,
let's talk about
the effect of
attacking an image search
on the cloud.
You can see that
this is a normal cat.
When the image search
on the cloud,
the result is
something I don't know how to translate.
It should be called
British long-haired cat.
It is similar to that.
It can be identified normally
when dealing with
our original sample.
The following is
an example of our attack.
Although there is a gap,
it won't affect your identification.
Notice that
the image I'm talking about
refers to this cat,
not the whole screenshot.
I'm talking about this cat.
You can see that
from the perspective of
although there is a gap between
it won't affect your identification.
However,
for image sorting,
we don't compare each image.
Instead,
we make the image into a label.
For example,
this is the result of a cat.
After that,
the image of the cat
will be transferred to you.
Therefore,
our attack on the cloud
will be successful in
attacking the image search.
In fact,
we don't know
what kind of image sorting model Google uses.
The whole process is pure black box.
Let's summarize.
The first point is that
through our experiment,
we have proven that
keeping the model in the cloud
is actually a wrong sense of security.
At least in the task of image sorting,
we have successfully achieved the attack.
In the case of the model,
I don't know at all.
After training with AutoDL,
our attack is generally
more than 80% effective.
This is a very good result.
It's a pure black box.
The key point is that
we have only checked the number of times.
The second point is
that our attack
has a very strong transparency.
It doesn't mean that
if I don't have the same open source model,
my attack will fail.
From the common recognition of cats and dogs
to the recognition of vehicles
to image sorting,
they are all based on
the model of IMAGINA.
So this attack has a very strong transparency.
In short,
I want to tell you that
if you put your model in the cloud,
even though you think that
you have a lot of image sorting,
you can also think that
no one knows about your model,
you can also think that
other people visit you a lot,
and it's impossible for someone to spend 100 yuan
to visit you 10,000 times
to create a picture of a cat and dog.
Keeping your model in the cloud
is a wrong sense of security.
Okay, I'm done with my presentation.
Thank you.
